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/  ABSTRACT 

The  development  of  stimulus  selectivity  in  primary  sensory  cortex  of  higher 
vertebrates  is  considered  in  a  general  mathematical  framework.  A  synaptic 
evolution  scheme  of  a  new  kind  is  proposed  in  which  incoming  patterns  rather 
than  converging  afferents  compete.  The  change  in  ^he  efficacy  of  a  given 
synapse  depends  not  only  on  Instantaneous  pre  and  postsynaptlc  activities 
but  also  on  a  slowly  varying  time-averaged  value  of  the  postsynaptlc  activity. 
Assuming  an  appropriate  nonlinear  form  for  this  dependence,  development  of 
selectivity  is  obtained  under  quite  general  conditions  on  the  sensory 
environment.  One  does  not  require  nonlinearity  of  the  neuron's  integrative 
power  nor  does  one  need  to  assume  any  particular  form  for  intracortical 
circuitry.  This  is  'fi^st^illustrated  in  simple  cases,  e.g.  when  the 
environment  consists  of  only  two  different  stimuli  presented  alternately 
in  a  random  manner.  The  following  formal  statement  then  holds:  the  state 
of  the  system  converges  with  probability  1  to  points  of  maximum  selectivity 
in  the  state  space.  We  next  consider  the  problem  of  early  development 
of  orientation  selectivity  and  binocular  interaction  in  primary  visual 
cortex.  Giving  the  environment  an  appropriate  form,  we  obtain  orientation 
tuning  curves  and  ocular  dominance  comparable  to  what  is  observed  in 
normally  reared  adult  cats  or  monkeys.  Simulations  with  binocular 


input  and  various  tjpes  of  normal  or  altered  environments  show  good 
agreement  with  'tb«^televant  experimental  data.  Experi  'jnts  are  suggested 
that  could  further  test  our  theory. 


ABSTRACT 


The  development  of  stimulus  selectivity  In  primary  sensory  cortex  of 
higher  vertebrates  Is  considered  In  a  general  mathematical  framework.  A 
synaptic  evolution  scheme  of  a  new  kind  Is  proposed  In  which  incoming 
patterns  rather  than  converging  afferents  compete.  The  change  In  the 
efficacy  of  a  given  synapse  depends  not  only  on  instantaneous  pre  and 
postsynaptic  activities  but  also  on  a  slowly  varying  time-averaged  value  of 
the  postsynaptic  activity.  Assuming  an  appropriate  nonlinear  form  for  this 
dependence,  development  of  selectivity  is  obtained  under  quite  general  con¬ 
ditions  on  the  sensory  environment.  One  does  not  require  nonlinearity  of 
the  neuron's  integrative  power  nor  does  one  need  to  assume  any  particular 
form  for  intracortlcal  circuitry.  This  is  first  illustrated  in  simple 
cases,  e.g.  when  the  environment  consists  of  only  two  different  stimuli 
presented  alternately  in  a  random  manner.  The  following  formal  statement 
then  holds:  the  state  of  the  system  converges  with  probability  1  to  points 
of  maximum  selectivity  in  the  state  space.  We  next  consider  the  problem 
of  early  development  of  orientation  selectivity  and  binocular  interaction 
in  primary  visual  cortex.  Giving  the  environment  an  appropriate  form,  we 
obtain  orientation  tuning  curves  and  ocular  dominance  comparable  to  what  is 
observed  in  normally  reared  adult  cats  or  monkeys.  Simulations  with  bi¬ 
nocular  input  and  various  types  of  normal  or  altered  environments  show 
good  agreement  with  the  relevant  experimental  data.  Experiments  are 
suggested  that  could  further  test  our  theory. 
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I.  INTRODUCTION 

It  has  been  known  for  some  time  that  sensory  neurons  at  practically  all 
levels  display  various  forms  of  stimulus  selectivity.  They  nay  respond 
preferentially  to  a  tone  of  a  given  frequency,  a  light  spot  of  a  given  color, 
a  light  bar  of  a  certain  length,  retinal  disparity,  orientation,  etc... 

We  might,  therefore,  regard  stimulus  selectivity  as  a  general  property  of 
sensory  neurons,  and  conjecture  that  the  development  of  such  selectivity 
obeys  some  general  rule.  Most  attractive  Is  the  Idea  that  some  of  the 
mechanisms  by  which  selectlvltles  develop  in  embryonic  or  early  post  natal 
life  are  sufficiently  general  to  allow  a  unifying  theoretical  treatment. 

In  the  present  paper  we  attempt  to  construct  such  a  mathematical  theory  of 
the  development  of  stimulus  selectivity  in  cortex.  It  is  based  on:  1)  an 
elementary  definition  of  a  general  index  of  selectivity,  2)  stochastic 
differential  equations  proposed  as  a  description  of  the  evolution  of  the 
strengths  of  all  synaptic  junctions  onto  a  given  cortical  neuron. 

The  ontogenetic  development  of  the  visual  system,  particularly  of 
higher  vertebrates,  has  been  very  extensively  studied.  Since  the  work  of 
Hubei  and  Wlesel  (1959,  1962)  it  has  been  known  that  almost  all  neurons  in 
the  primary  visual  cortex  (area  17)  of  the  normally  reared  adult  cat  are 
selective;  they  respond  in  a  precise  and  sometimes  highly  tuned  fashion 
to  a  variety  of  features — in  particular  to  bars  or  edges  of  a  given  orientation 
and/or  moving  in  a  given  direction  through  their  receptive  fields.  Further 
work  has  shown  that  the  response  characteristics  of  these  cortical  cells 
strongly  depend  on  the  visual  environment  experienced  by  the  animal  during 
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a  critical  period  extending  roughly  from  the  3rd  to  the  15th  week  of 
postnatal  life,  (see  for  example  Hubei  and  Wlesel,  1965;  Blakemore  and 
Van  Sluyters,  1975;  Bulsseret  and  Imbert,  1976;  Fregnac  and  Imbert,  1978; 
and  Fregnac,  1979).  Although  these  experiments  show  that  visual  experience 
plays  a  determining  role  in  the  development  of  selectivity  the  precise 
nature  of  this  role  is  still  a  matter  of  controversy. 

Applying  our  general  ideas  to  the  development  of  orientation  selectivity 
and  binocular  interaction  in  area  17  of  cat  visual  cortex,  we  obtain  a  theory 
based  on  a  single  mechanism  of  synaptic  modification  that  accounts  for  the 
great  variety  of  experimental  results  on  monocular  and  binocular  experience 
in  normal  and  various  altered  visual  environments.  In  addition  we  obtain 
some  new  predictions. 

It  is  known  that  various  algorithms  related  to  Hebb's  principle  of  synaptic 
learning  (Hebb;  i9A9)  can  account  for  the  formation  of  associative  and  distri¬ 
buted  memories  (see  for  example  Marr,  1969;  Brindley,  1969;  Anderson,  1970 
and  1972;  Cooper,  1973;  Kohonen,  1977).  We  therefore  suggest  that  it  may  be 
the  same  fundamental  mechanism,  accessible  to  detailed  experimental  investiga¬ 
tion  in  primary  sensory  areas  of  the  nervous  system  which  is  also  responsible 
for  some  of  the  higher  forms  of  central  nervous  system  organization. 

In  sections  II-IV,  our  ideas  are  presented  in  general  form,  section  V  is 
devoted  to  the  development  of  orientation  selectivity  primarily  in  a  normal 
visual  environment,  whereas  in  section  VI  it  is  shown  that  our  assumptions 
also  account  for  normal  or  partial  development  of  orientation  selectivity 
and  binocularity  in  various  normal  or  altered  visual  environments. 
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II.  PRELIMINARY  REMARKS  AND  DEFINITIONS 

Notation 

We  simplify  the  description  of  the  dynamics  of  a  neuron  by  choosing  as 

variables  not  the  Instantaneous  incoming  time-sequence  of  spikes  in  each 

afferent  fiber,  the  instantaneous  membrane  potential  of  the  neuron  or  the 

time-sequence  of  outgoing  spikes,  but  rather  the  pre  and  postsynaptlc 

firing  frequencies.  These  may  be  thought  of  as  moving  time-averages  of 

3 

the  actual  Instantaneous  variables,  where  the  length  of  the  averaging 
interval  is  of  the  order  of  magnitude  of  the  membrane  time-constant  t  . 
Throughout  this  paper,  these  firing  frequencies  are  used  as  instantaneous 
variables.  This  formal  neuron  is  thus  a  device  which  performs  spatial 
integration  (it  integrates  the  signals  impinging  all  over  the  soma  and 
dendrites)  rather  than  spatio-temporal  integration:  the  output  at  time  t 
is  a  function  of  the  input  and  synaptic  efficacies  at  t,  independent  of 
past  history. 

A  synaptic  efficacy  m^  characterizes  the  net  effect  of  presynaptic  neuron 
i  on  the  postsynaptlc  neuron  (in  most  of  the  paper  only  one  postsynaptlc 
neuron  is  considered) .  This  effect  may  be  mediated  through  a  complex  system 
including  perhaps  several  interneurons  some  of  which  are  excitatory, 
other  inhibitory.  The  resulting  "ideal  synapse"  (Nass  and  Cooper,  1975) 
may  thus  be  of  either  sign  depending  on  whether  the  net  effect  is  excitatory 
or  inhibitory;  it  may  also  change  sign  during  development. 


EiL 
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A  further  simplification  is  to  asstaae  that  the  integrative  power  of  the 
neuron  is  a  linear  function^  that  is: 


c(t)  =  (t)dj (t) 


(1) 


where  c(t)  is  the  output  at  time  t,  the  efficacy  of  the  j-th  synapse 


at  time  t,  ‘Ij(t)  is  the  j-th  component  of  the  input  at  time  t  (the  firing 


frequency  of  the  j-th  presynaptic  neuron)  and  denotes  summation  over  j,  i.e. 


over  all  presynaptic  neurons.  We  can  then  write: 


m(t)  =  (mj^(t),  m2(t),  ....  injg(t)) 
d(t)  =  (d^(t),  d2(t),  d^(t)) 

c(t)  =  m(t)-d(t)  (2) 


m(t)  and  d  (t)  are  real-valued  vectors,  of  the  same  dimension,  N,  i.e.,  the 
number  of  ideal  synapses  onto  the  neuron,  and  c(t)  is  the  inner  product  (or 
"dot  product")  of  m(t)  and  d(t).  The  vector  m(t),  i.e.  the  array  of  synaptic 
efficacies  at  time  t,  is  called  the  state  of  the  neuron  at  time  t.  (Note 
that  c(t)  as  well  as  all  components  of  d(t)  represent  firing  frequencies 
that  are  measured  from  the  level  of  average  spontaneous  activity;  thus 
they  might  take  negative  as  well  as  positive  values;  mj(t)  is  dimensionless.) 


The  precise  form  of  the  integrative  power  is  not  essential:  our  results 
remain  unchanged  if  for  instance  c(t)  =  S (m(t) •d(t) )with  S  a  positive-valued 
sigmoid-shaped  function  (see  Bienenstock,  1980) .  This  is  in  contrast  to 
other  work  (e.g.  Von  der  Malsburg,  1973)  that  does  require  nonlinear  in¬ 
tegrative  power  (see  Appendix  B) . 


Selectivity 

It  is  common  usage  to  estimate  the  orientation  selectivity  of  a  single 
visual  cortical  neuron  by  measuring  the  half-width  at  half-height — or  an 
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equivalent  quantity — of  its  orientation  tuning  curve.  The  selectivity 
is  then  measured  with  respect  to  a  parameter  of  the  stimulation,  namely 
the  orientation,  which  takes  on  values  over  an  interval  of  180°.  In  the 
present  study,  various  kinds  of  inputs  are  considered,  e.  g.  formal  Inputs 
with  a  parameter  taking  values  on  a  finite  set  of  points,  rather  than  a 
continuous  interval.  It  will  then  be  useful  to  have  a  convenient  general 
index  of  selectivity,  defined  in  all  cases.  We  propose  the  following: 


yv,  mean  response  ot{/^ with  respect  to  d 

Sel .  (c/T)  =  1 - - - - !L 

a  maximum  response  of./7/7with  respect  to 


With  this  definition,  selectivity  is  estimated  with  respect  to,  or  ^  an 
environment  for  the  neuron,  that  is,  a  random  variable  d  that  takes  on  values 
in  the  space  of  inputs  to  the  neuron  .  The  variable  represents  a  random 
input  to  the  neuron,  and  is  characterized  by  its  probability  distribution  that 
may  be  discrete  or  continuous.  (During  normal  development,  the  input  to  the 
neuron  (or  neuronal  network)  is  presumably  distributed  uniformly  over  all 
orientations.  In  abnormal  rearing  conditions  (e.g.  dark  reared)  the  input 
during  development  could  be  different  from  the  input  for  measuring  selectivity. 
How  this  should  be  translated  in  the  formal  space  R  will  be  discussed  in 
section  V),  This  distribution  defines  an  environment,  mathematically  a 


random  variable  jJ.  Selectivity  is  estimated  (before,  or  after  development) 

with  respect  to  this  same  environment^.  Obviously,  Sel,  always  falls 

/  ^ 

between  0  and  1,  and  the  higher  the  selectivity  of  in  jJ,  the  closer 


is  to  1. 
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When  applied  to  the  formal  neuron  In  state  m  definition  (3)  gives: 

E[m  • 

Selj(m)  *  1  -  - - - - ^ 

ess  sup(m  •  d) 

N 

where  d  is  any  R  -valued  random  variable  (the  formal  environment  for  the 
neuron).  The  symbol  E  [...]  stands  for  "expected  value  of  ..."  (i.e.  mean 
value  with  respect  to  the  distribution  of  (^)  and  "ess  sup  of  ..."  (essential 
supremum)  is  equivalent  to  "maximum  of  ..."  in  most  common  applications.  This 
is  Illustrated  in  Fig.  1. 


III.  MODIFICATION  OF  CORTICAL  SYNAPSES 


The  various  factors  that  influence  synaptic  modification  may  be  divided 
broadly  into  two  classes — those  dependent  on  global  and  those  dependent 
on  local  information.  Global  Information  in  the  form  of  chemical  or 
electrical  signalling  presumably  influences  in  the  same  way  most  (or  all) 
modifiable  junctions  of  a  given  type  in  a  given  area.  Evidence  for  the 
existence  of  global  factors  that  affect  development  may, for  instance, be  found 
in  Kasamatsuand  Pettigrew  (1976,  1979),  Singer  (1979,1980),  and  Buisseret 
et  al.  (1978).  On  the  other  hand,  local  information  available  at  each 
modifiable  synapse  can  influence  each  junction  in  a  different  manner.  In 
this  paper  we  are  primarily  Interested  in  the  effect  of  local  information  on 
the  development  of  selectivity. 

An  early  proposal  as  to  how  local  information  could  affect  synaptic 
modification  was  made  by  Hebb  (1949).  His,  now  classical,  principle  was 
suggested  as  a  possible  neurophysiological  basis  for  operant  conditioning; 
"when  an  axon  of  cell  A  is  near  enough  to  excite  a  cell  B  and  repeatedly  or 
persistently  takes  part  in  firing  it,  some  growth  process  or  metabolic  change 
takes  place  in  one  or  both  cells  such  that  A’s  efficiency,  as  one  of  the  cells 
firing  B,  is  increased".  Thus  the  increase  of  the  synaptic  strength 
connecting  A  to  B  is  dependent  upon  the  correlated  firing  of  A  and  B.  Such 
a  correlation  principle  has  inspired  the  work  of  many  theoreticians  on 
various  topics  related  to  learning,  associative  memory,  pattern  recognition, 
organization  of  neural  mappings  (retinotopic  projections)  and  development 
of  selectivity  of  cortical  neurons. 

It  is  fairly  clear  that  in  order  to  actually  use  Hebb's  principle  one  must 
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state  conditions  for  synaptic  decrease  as  specific  as  those  for  synaptic 
increase:  if  synapses  are  allowed  only  to  increase,  all  synapses  will 
eventually  saturate;  no  information  will  be  stored  and  no  selectivity  will 
develop  (see  for  example  Sejnowski,  1977a,  b) .  What  is  required  is  thus  a 
complementary  statement  to  Hebb's  principle  giving  conditions  for  synaptic 
decrease. ^ 

Such  statements  usually  have  resulted  in  a  form  of  synaptic  competition. 
Consider,  for  example,  one  that  was  proposed  by  Stent  (1973):  "when  the 
presynaptic  axon  of  cell  A  repeatedly  and  persistently  fails  to  excite  the 
postsynaptic  cell  B  while  cell  B  is  firing  under  the  influence  of  other  pre¬ 
synaptic  axons,  metabolic  changes  take  place  in  one  or  both  cells  such  that 
A’s  efficiency,  as  one  of  the  cells  firing  B,  is  decreased".  According  to 
Stent's  principle,  the  increase  of  the  strength  of  certain  synapses  onto 
neuron  B  is  accompanied  by  simultaneous  decrease  of  the  strength  of  other 
synapses  onto  the  same  neuron  B.  There  thus  occurs  a  spatial  competition 
between  convergent  afferents.  A  competition  mechanism  of  this  kind  provides 
a  qualitative  explanation  of  some  experimental  results  on  cortical  develop¬ 
ment  (e.g.  monocularly  deprived  animals  (Stent,  1973))  as  well  as  some 
aspects  of  certain  more  complex  deprivation  paradigms  such  as  those  recently 
reported  by  Rauschecker  and  Singer  (1981) . 
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In  the  present  work,  we  present  a  mechanism  of  synaptic  modification 
that  results  In  a  temporal  competition  between  Input  patterns  rather 
than  a  spatial  competition  between  different  synapses.  With  this 
mechanism,  whether  synaptic  strength  Increases  or  decreases  depends  upon 
the  magnitude  of  the  postsynaptlc  response  as  compared  with  a  variable 
modification  threshold.  We  show  that  this  can  account  quantitatively  in 
a  more  powerful  way  for  Increases  and  decreases  In  selectivity,  as  well 
as  a  great  variety  of  other  experimental  results  in  diverse  rearing 
conditions. 


We  propose  that  the  change  of  the  j-th  synapse's  strength  at  that  time  t  obeys 
the  following  rule: 

(t)  «  (t>(^(t)))^^(t)  -  em^(t)  (4) 

where  is  a  scalar  function  of  the  pofetsynaptic  activity,  c(t) ,  that 

changes  sign  at  a  value,  9^^,  of  the  output  called  the  modification  threshold: 

<|>(c)<0  for  c  <9„,  <t>(c)>0  for  c>0„ 

'v  '\i  n  %  '\j  n 


The  term,  -€m(t),  produces  a  uniform  decay  of  all  junctions;  this  in  most 
cases  does  not  affect  the  behavior  of  the  system  if  t  is  small  enough.  How¬ 
ever,  as  will  be  seen  later,  it  is  Important  in  some  situations.  Other  than 
this  uniform  decay,  the  vector  is  driven  in  the  direction  of  the  input 
if  the  output  is  large  (above  9^^)  or  opposite  to  the  direction  of  the  input 

if  the  output  is  small  (below  9„).  As  in  hebblan  modification,  when  d  >0  and 

M  '\.J 

is  large  enough  m^  Increases.  However,  when  ^^>0  and  c  is  not  large  enough, 
TOj  decreases.  We  may  regard  this  as  a  form  of  temporal  competition  between 
incoming  patterns. 


The  idea  of  such  a  modification  scheme  was  5r produced  in  Cooper  et  al.  (1979). 

Their  use  of  a  constant  threshold  0,.,  however,  re^nl  :ed  a  certain  lack  of 

M 
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robustness  of  the  system:  the  response  to  all  patterns  could  slip 
below  9„  and  then  decrease  to  zero.  In  the  absence  of  lateral  Inhibition 
between  neurons,  the  response  might  increase  to  more  than  one  pattern  leading 
to  stable  states  with  maximal  response  to  more  than  one  pattern. 

In  this  paper,  we  will  see  that  making  an  appropriate  choice  for  allows 

correct  functioning  under  quite  general  conditions  and  provides  remarkable 
noise-tolerance  properties. 

In  our  threshold  modification  scheme,  the  change  of  the  j-th  synapse's  strength 
is  written  as  a  product  of  two  terms,  the  presynaptic  activity  (t) ,  and  a 
function,  ^()^(t),  ^(t))  of  the  postsynaptic  variables,  the  output,  (^(t) 
and  the  average  output  ^(t).  Making  use  of  c(t)  in  the  evolutive  power  of  the 
neuron  is  a  new  and  essential  feature  of  this  work.  It  is  necessary  in  order 
to  allow  both  boundedness  of  the  state  and  efficient  threshold  modification. 

Neglecting  the  uniform  decay  term,  for  the  moment,  (€*0),  in  vector  notation 
we  have 

l^(t)  -  <K(^(t),  ^(t))  jjCt)  (5) 

This,  together  with  eq.  2,  yields: 

^(t)  »  <K?g(t)  •  s^(t),  jg(t)  •  d)  jJ(t)  (6) 

The  crucial  point  in  the  choice  of  the  function  4)(c,c)  is  the  determination 
of  the  threshold  l»e*t  the  value  of  c  at  which  (}i(c,c)  changes  sign.  A 

candidate  for  Q^Ct)  is  the  average  value  of  the  postsynaptic  firing 
rate,  !^(t).  The  time  average  is  meant  to  be  taken  over  a  period 


T  preceding  t  much  longer  than  the  membrane  time-constant  t  so 
Chat  c(c)  evolves  on  a  much  slower  time-scale  than  c(t).  This  can 
usually  be  approximated^  by  averaging  over  the  distribution  of  Inputs 
for  a  given  state  m(c) 

c(t)  =>  m(t)*d 

This  results  in  an  essential  feature,  the  instability  of  low  selectivity 
points,  as  will  now  be  Informally  shown. 

1  K 

As  a  simplification,  assume  Chat  takes  on  K  discrete  values  d  ,  ...,  d  with 

equal  probability  and  let  m^  be  a  low-selectivity  equilibrium  point. 

Equilibrium  means  that  the  average  E[(()(m  ♦  d)d]  ■  (l/K)E%(m  •  d^)d^  is 

o  'v  'v  i»l  ® 

the  null  vector,  while  low-selectivity  means  that  for  most  of  the  d^  s, 

say  all  of  them,  the  output  m^‘  d^  is  positive,  and  m  •  d^  *  m  •  d^  for 

o  O  o 

all  i  and  J.  It  follows  that  for  all  1, 

m^-  =  E[m^  •  d]=  0j,(m^)  hence  (<i(m^  •  d^)=^  0. 

Consider  now  a  small  perturbation  from  equilibrium:  m  ■  m^  +  x.  It  is 
likely  that  x  will  "favor"  some  of  the  d^’s  to  the  detriment  of  others: 
a  favored  input  is  a  d^  roughly  pointing  in  the  direction  of  x,  which  there¬ 
fore  satisfies  ra  •  d^>  0jj(m),  hence  ,|.(m  •  d^)>0,  while  for  the  d^’s  directed 
opposite  to  X  one  has  <(i(m  •  d^)<0.  Thus,  when  a  favored  input  d^  comes  in. 


the  state  m  moves  in  Its  direction  (eq.  6  with  <p(c)>0),  while  when  a 
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non-favored  Input  comes  In,  It  moves  In  the  opposite  direction.  In  sunmary, 
whatever  the  Input  d^,  the  state  moves  in  the  direction  of  the  initial 
perturbation  x,  away  from  m^,  which  is  therefore  an  unstable  equilibrium 
point. 


We  may  now  state  that  if  stable  equilibrium  points  exist  in  the  state  space, 
they  are  of  high  selectivity.  However  do  such  points  exist  at  all?  The 
answer  is  generally  yes  provided  that  the  state  is  bounded  from  the  origin  and 
from  infinity.  These  conditions.  Instability  of  low-selectivity  equilibria 
as  well  as  boundedness,  are  fulfilled  by  a  single  function  iKc,  c)  if  we 

define  0..(t)  to  behave  as  a  nonlinear  function  of  c(t),  for  example 

M  -  -- 

a  power.  The  exponent  should  then  be  larger  than  1.  The  final  requirement 
on  ((>(c,c)  thus  reads: 


sign  ((i(c,c)  =  sign  (c-(^)^c)  for  c>0 

(7) 

(ji(0,c)  ■  0  for  all  c 


where  Cq  and  p  are  two  fixed  positive  constants.  The  threshold 
0jj(c)  =  (c/c^)^c  thus  serves  two  purposes;  allowing  threshold 
modification  when  c  c^  as  well  as  driving  the  state  from  regions  such 
that  c  <<  Cq  or  c  >>  Cq.  Equation  (7)  is  Illustrated  in  Fig.  2.  The 
process  of  synaptic  growth,  starting  near  0  to  eventually  end  in  a  stable 
selective  state,  may  be  described  as  follows.  Initially,  c  «  Cq  hence 


(^(c,  c)  >  0  for  all  Inputs  In  the  environment:  the  responses to  all  Inputs 
grow.  With  this  growth  c  Increases,  thus  increasing  0^.  Now  some  inputs 
result  in  postsynaptic  responses  that  exceed  0^^,  while  others— those  whose 
direction  is  far  away  (close  to  orthogonal)  from  the  favored  Inputs — give 
a  response  less  than  Q^.  The  response  to  the  former  continues  to  grow  while 
the  response  to  the  latter  decays.  This  results  in  a  form  of  competition 
between  incoming  patterns  rather  than  competition  between  synapses.  The 
response  to  unfavored  patterns  decays  until  it  reaches  0,  where  it  stabilizes, 
for  (>(0,c)  *  0  for  any  c  (equation  7).  The  response  to  favored  patterns  grows 
until  the  mean  response  c  is  high  enough,  and  the  state  stabilizes.  This  occurs 
in  spite  of  the  fact  that  many  complicated  geometrical  relationships  may  exist 
between  different  patterns,  i.e.  that  they  are  not  orthogonal  since  different 
patterns  may  and  certainly  do  share  common  synapses. 

It  should  be  stressed  that  this  is  only  one  solution  among  many  possible  others. 
The  purpose  of  these  equations  is  to  provide  a  scheme  (rather  than  a  unique 
differential  equation)  that  is  biologically  feasible,  yet  defined  well  enough 
for  mathematical  and  numerical  analysis.  Considerable  freedom  is  left  in  the 
exact  choice  of  the  function  <!>  since  p  and  c^  may  be  almost  arbitrarily  chosen 
and,  once  this  is  done,  the  only  requirements  are  on  the  sign  of  (|i. 
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IV.  MATHEMATICAL  RESULTS 

The  behavior  of  system  (6)  depends  critically  on  the  environment,  that  Is, 
on  the  distribution  of  the  stationary  stochastic  process  jJ.  Two  classes 
of  distributions  may  be  considered: 

1  K 

-  discrete  distributions:  K  possible  inputs  d  .  d  . 

These  will  generally  be  assumed  to  occur  with  the  same  probability  1/K. 

The  process  is  then  a  jump  process  which  randomly  assumes  new  values  at 
each  time  Increment.  The  vector  ^  is  (roughly)  a  Markov  process. 

-  continuous  distributions:  in  the  present  work,  the  only 

continuous  distribution  that  will  be  considered  is  a  uniform  distribution 

N 

over  a  closed  1-parameter  curve  in  the  input  space  R  (section  V) . 

Although  the  principles  underlying  the  convergence  to  selective  states  are 
intuitively  fairly  simple  (see  preceding  section) ,  mathematical  analysis  of 
the  system  is  not  entirely  straightforward,  even  for  the  simplest  Mathe¬ 
matical  results,  obtained  only  for  certain  discrete  distributions,  are  of  two 
types:  1)  equilibrium  points  are  locally  stable  if  and  only  if  they  are  of 
highest  available  selectivity  with  respect  to  the  given  distribution  of 
2)  given  any  initial  value  of  m  in  the  state  space,  the  probability  that  ^(t) 
converges  to  one  of  the  maximum  selectivity  fixed  points  as  t  goes  to  infinity 
is  1.  Results  Of  the  Second  type  are  much  stronger,  and  require  a  tedious 
geometrical  analysis.  Results  are  stated  here  in  a  somewhat  simplified  form 
(obvious  requirements  of  a  very  mathematical  character  are  omitted) .  For  exact 
statements  and  proofs,  the  reader  is  referred  to  Blenenstock  (1980). 

We  first  study  the  simple  case  where  takes  on  values  of  only  two  possible 
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1  2 

input  vectors  d  and  d  ,  that  occur  with  the  same  probability; 

-  d^l  -  P[jJ  -  d^]  =  1/2. 

Whatever  the  real  dimension  N  of  the  system  it  reduces  to  two  dimensions. 

1  2 

^y  component  of  ^  outside  the  linear  subspace  spanned  by  d  and  d  will 
eventually  decay  to  0  due  to  the  uniform  decay  term.) 

It  follows  immediately  from  the  definition  that  the  maximum  value  of 

in  the  state  space  is  1/2.  It  is  reached  for  states  m  which  give  null  response 

1  12 
when  d  comes  in  (i.e.  are  orthogonal  to  d  )  but  positive  response  for  d  —  or 

vice  versa.  Minimum  selectivity,  namely  0,  is  obtained  for  states  m  such  that 
1  2 

m  •  d  ■  m  •  d  .  Equilibrium  states  of  both  kinds  indeed  exist: 

12  1 
Lemma  1:  Let  d  and  d  be  linearly  independent  and  satisfy  P[jlJ  *  d  ]  • 

2 

PfjJ  *  d  ]  ■  1/2.  Then  for  any  <!>  satisfying  eq.  (7),  eq.  (6)  admits  exactly  4 

fixed  points,  m®,  m^,  m^  and  m^*^  with:  Sel.(m®)  =  Sel,(m^’^)  *  0,  and  Sel.(m^)  • 

K  K 

2  V.  ^ 

Sel,(m  )  ■  1/2.  (Here  the  superscripts  indicate  which  of  the  d  are  not  orthogonal 

p  - 

to  m.  (m^  is  the  origin).  Thus  for  Instance  m^*  d^  >  0,  m^  •  d^*  0.) 

The  behavior  of  eq.  6  depends  on  the  geometry  of  the  Inputs,  in  the  present 
1  2 

case  on  cos(d  ,d  ).  The  crucial  assumption  that  is  needed  here  is  that 
1  2 

cos(d  ,d  )^0.  This  Is  a  reasonable  assumption  which  is  obviously  satisfied  If 
all  components  of  the  Inputs  are  positive,  as  is  assumed  in  some  models  (Von  der 
Malsburg,  1973;  Perez  et  al.,  1975).  We  may  then  state  the  following: 

1  2 

Theorem  1.  Assume  that  in  addition  to  the  conditions  of  Lemma  1,  cos(d  ,  d  )  >  0. 

0  12  12 

Then  m  and  m  *  are  unstable,  m  and  m  are  stable,  and  whatever  its  initial 
value,  the  state  of  the  system  converges  almost  surely  (i.e.,  with  probability  1) 
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1  2 
either  to  m  or  to  m  . 

Theorem  1  is  the  basic  result  in  the  2-dimensional  setting:  it  characterizes 
evolution  schemes  based  on  competition  between  patterns,  saying  that  the  state 
eventually  reaches  maximal  selectivity  even  when  the  two  input  vectors  are 
very  close  to  one  another.  Obviously  this  requires  that  some  of  the  synaptic 
strengths  be  negative  since  the  neuron  has  linear  integrative  power.  Inhibitory 
connections  are  thus  necessary  to  obtain  selectivity  (see  also  section  V  below) . 
Some  selectivity  is  also  realizable  with  no  inhibitory  connections — not  even 
"intracortlcal"  ones — if  the  integrative  power  is  appropriately  nonlinear. 
However,  whatever  the  nonlinearity  of  the  integrative  power,  theorem  1  could 
not  hold  for  evolution  equations  based  on  comeptition  between  converging 
afferents  (see  Appendix  B) . 

In  theorem  1,  we  have  a  discrete  sensory  environment  which  consists  of 
exactly  two  different  stimuli — a  situation,  although  simple  mathematically, 
not  often  encountered  in  nature.  It  may  however,  very  well  correspond  to 
a  visual  environment  restricted  to  only  horizontally  and  vertically  oriented 
contours,  present  with  equal  probability.  Theorem  1  then  predicts  that 
cortical  cells  will  develop  a  selective  response  to  one  of  the  two  orientations, 
with  no  preference  for  either  (other  than  what  may  result  from  initial 
connectivity).  Thus,  on  a  large  sample  of  cortical  cells,  one  should  expect 
as  many  cells  tuned  to  the  horizontal  orientation  as  to  the  vertical  one.  (So 

far,  no  assumption  is  made  on  Intracortlcal  circuitry.  See  Appendix  D) . 

The  proof  of  theorem  1  is  based  on  the  existence  of  trap  regions  around  each  of 
of  the  selective  fixed  points: 
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Theorem  2;  Under  the  same  conditions  as  in  Theorem  1,  there  exists 

12  12  12 
around  m  (m  )  a  region  F  (F  )  such  that  once  the  state  enters  F  (F  ) , 

1  2 

it  converges  almost  surely  to  m  (m  ) . 

The  meaning  of  theorem  2  is  the  following;  once  m(t)  has  reached  a  certain 

'V 

selectivity;  it  cannot  "switch"  to  another  selective  region.  Applied  to  cortical 
cells  in  a  patterned  visual  environment,  this  means  that  once  they  become 
sufficiently  committed  to  certain  orientations,  they  will  remain  committed  to 
those  orientations,  (provided  that  the  visual  environment  does  not  change), 
becoming  more  selective  as  they  stabilize  to  some  maximal  selectivity'.  Theorems  1 
and  2  are  Illustrated  in  Figure  3. 

1  2 

It  is  worth  mentioning  that  when  cos(d  ,  d  )<0,  the  situation  is  much  more 

complicated:  trap  regions  don’t  necessarily  exist  and  periodic  asymptotic 

behavior,  i.e.,  limit  cycles,  may  occur,  bifurcating  from  the  stable  fixed 
1  2 

points  when  cos(d  ,d  )  becomes  too  negative  (see  Bienenstock,  1980). 

We  now  turn  to  the  case  where  takes  on  K  values.  The  following  is  easily 
obtained: 

12  K 

Lemma  2;  let  d  ,  d  ,  . . . ,  d  be  linearly  independent  and  satisfy 
1  k 

P[jJ  *  d  ]  *  ...  *  P[(J  »  d  ]  =  1/K.  Then,  for  any  <(i  satisfying  equation  (7), 

V 

eq.  (6)  admits  exactly  2  fixed  points  with  selectlvities  0,  1/K,  2/K,  ..., 
(K-l)/K.  There  are  K  fixed  points  m^,  ...,  m^  of  selectivity  (K-l)/K. 

Obviously,  (K>l)/K  is  also  the  maximum  possible  selectivity  with  respect 
to  It  means  a  positive  response  for  one  and  only  one  of  the  inputs.  The 
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situation  Is  now  much  more  complicated  that  what  Is  was  with  only 
2  Inputs:  It  Is  not  obvious  whether  In  all  cases  assuming  that  all  the 
cosines  between  inputs  are  positive  is  sufficient  to  yield  stability  of  the 
maximum  selectivity  fixed  points.  However,  we  may  state  the  following: 

1  K 

Theorem  3;  Assume,  in  addition  to  the  conditions  of  Lemma  2,  that  d  ,  ...,  d 
are  all  mutually  orthogonal  or  close  to  orthogonal.  Then  the  K  fixed  points 
of  maximum  selectivity  are  stable,  and,  whatever  Its  Initial  value,  the  state 
of  the  system  converges  almost  surely  to  one  of  them. 

The  proof  of  theorem  3  also  involves  trap  regions  areound  the  K  maximally 
selective  fixed  points,  and  the  analog  of  theorem  2  Is  true  here. 

Although  the  general  case  has  not  yet  been  solved  analytically,  as  will  be 

seen  In  the  next  section,  computer  simulations  suggest  that  for  a  fairly  broad 

1  i  IK 

range  of  environments  If  d  •  d*^  5,0,  even  if  d  ,  ....  d  are  far  from  being 
mutually  orthogonal,  the  K  fixed  points  of  maximum  selectivity  are  stable. 

Simulations  suggest  further  (see  for  Instance  Fig.  4b)  that  even  if  the 
1  K 

d  .  . . . ,  d '  are  not  linearly  Independent  and  are  very  far  from  being  mutually 
orthogonal,  the  asymptotic  selectivity  Is  close  to  its  maximum  value  with 
respect  to 
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V.  ORIENTATION  SELECTIVITY  AND  BINOCULAR  INTERACTION  IN  VISUAL  CORTEX 

We  now  apply  what  has  been  done  above  to  a  concrete  example,  orientation 
selectivity  and  binocular  interaction  in  primary  visual  cortex.  The 
ordinary  development  of  these  properties  in  mammals  depends  to  a  large 
extent  on  normal  functioning  of  the  visual  system  (i.e.  normal  visual 
experience)  during  the  first  few  weeks  or  months  of  post-natal  life.  This 
has  been  demonstrated  many  times  by  various  experiments,  based  mainly  on  the 
paradigm  of  rearing  the  animal  in  a  restricted  sensory  environment.  In  the 
next  two  sections,  it  Is  shown  how  equations  (4)-(7)  account  for  both  normal 
development  as  well  as  development  in  restricted  visual  environments. 

Consider  first  a  classical  test-environment  used  to  construct  the  tuning 
curve  of  cortical  neurons.  This  environment  consists  of  an  elongated  light 
bar  successively  presented  or  moved  in  all  orientations  —  preferably  in  a 
random  sequence  —  in  the  neuron's  receptive  field.  Thus  all  the  parameters 
of  the  stlmiilus  are  constant  except  one,  the  orientation,  which  is  uniformly 
distributed  on  a  cii'cularly  symmetric  closed  path.  We  assume  that  the 
retino-cortlcal  pathways  maps  this  family  of  stimuli  to  the  cortical  neuron's 
space  of  inputs  in  such  a  way  as  to  preserve  the  circular  synmetry  (as  defined 
below).  Thus,  the  typical  theoretical  environment  that  will  be  used  for 
constructing  the  formal  neuron's  tuning  curve  is  a  random  variable  uniformly 
distributed  on  a  circularly  symmetric  closed  one-parameter  family  of  points 
in  the  space  R^.  The  parameter  coding  orientation  in  the  receptive  field 
is,  in  principle,  continuous.  However,  for  the  purpose  of  numerical  simulations, 
the  distribution  is  made  discrete.  Thus  ^  takes  on  values  on  the  points  d^,  ...,d 

The  requirement  of  circular  symmetry  is  expressed  mathematically  as  follows: 
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1  K 

the  matrix  of  inner  products  of  the  vectors  d  ,  . . .  d  is  circular 
(i.e.,  each  row  is  obtained  from  its  nearest  upper  neighbor  by  shifting 
it  one  column  to  the  right)  and  the  rows  of  the  matrix  are  unlmodal.  A 
random  variable,jJ, uniformly  distributed  on  such  a  set  of  points  will  be, 
hereafter,  called  a  circular  environment.  Such  a  d  may  be  roughly  charac- 

'  —  —  —I  P  I  I  ■  ■  I  ■  ■  ■  nj 

terized  by  3  parameters:  N,  K  and  a  measure  of  the  mutual  geometrical 
closeness  of  the  d^'s,  for  instance  min  cos(d^,d^). 

We  are  now  faced  with  the  difficult  problem  of  specifying  the  stationary 
stochastic  process  that  represents  the  time-sequence  of  inputs  to  the 
neuron  during  development.  In  a  first  analysis,  there  is  no  choice  but 
oversimplifying  the  problem  by  giving  the  stochastic  process  exactly  the 
same  distribution  as  the  circular  d  defined  above.  In  doing  so,  we  assxme 

'\j 

that  development  of  orientation  selectivity  is  to  a  large  extent  Independent 
of  other  parameters  of  the  stimulus,  e.g.,  contrast,  shape,  position  in  the 
receptive  field,  retinal  disparity  for  binocular  neurons,  etc.  The  elementary 
stimulus  for  a  cortical  neuron  is  a  rectilinear  contrast  edge  or  bar.  Any 
additional  pattern  present  at  the  same  time  in  the  receptive  field  is  regarded 
as  random  noise.  (A  discussion  of  this  point  is  given  in  Cooper  et  al.  (1979)). 

Va.  NORMAL  MONOCULAR  INPUT 

The  behavior  of  a  monocular  system  in  circular  environments  is  investigated 
by  n'jmerically  simulating  equation  6  with  a  variety  of  circular  jJ's  and  functions 
^  satisfying  equation  7.  In  the  simulations  presented  here,  the  dimensions  of 
the  input  and  state  space  is  generally  N  -  37;  the  ntjmber  K  of  input  vectors 
varies  from  12  to  60.  (Various  kinds  of  functions  were  used:  some  were 
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stepwise  constant,  others  smooth,  bounded  or  unbounded.)  One  may 
reasonably  expect  the  system's  behavior  to  be  fairly  Independent  of  N  and 
K  if  these  are  high  enough.  However,  the  geometry  of  d  may  be  determining: 
if  the  d^'s  are  closely  packed  together  in  the  state  space,  i.e.,  if  min 
cos(d^,d^)  is  close  to  1,  convergence  to  selective  states  may  presumably 
be  difficult  to  achieve  or  even  impossible. 

Simulations  show  the  following  behavior: 

1.  The  state  converges  rapidly  to  a  fixed  point,  or 
attractor. 

2.  Various  such  attractors  exist.  For  a  given  and 
they  all  have  the  same  selectivity,  i^hich  is  close 
to  its  maximum  value  in  <|J. 

3.  The  asymptotic  tuning  curve  is  always  unimodal.  One 
may  thus  talk  of  the  preferred  orientation  of  an 
attractor. 

4.  There  exists  an  attractor  in  each  possible  orientation. 

5.  If  there  is  no  initial  preference,  all  orientations  have 
equal  probability  of  attracting  the  state.  CWhich  one 
will  become  favored  depends  on  the  exact  sequence  of 
inputs) .  This  does  not  hold  for  environments  which  are 
not  perfectly  circular,  at  least  for  a  single  neuron  system 
as  the  one  studied  here. 


Figs.  4a  and  4b  show  respectively  the  progressive  buildup  of  selectivity  and 
the  tuning  curve  when  the  state  has  virtually  stabilized. 
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In  suBsnary  then,  the  system  behaves  In  circular  environments  exactly 

as  we  might  have  expected  from  the  results  of  the  preceding  section.  One 

should  however  note  one  important  difference:  the  maximum  selectivity  for 

a  continuous  environment  cannot  be  as  simply  calculated  as  it  was  before. 

It  is  only  when  d  is  uniformly  distributed  on  K  linearly  independent  vectors 

K— 1 

that  we  know  that  max  Sel,(m)  =  - —  (Lemma  2).  Theorem  3  tells  us  that  if 
in  addition  the  vectors  are  nearly  orthogonal  to  one  another,  this  selectivity 
is  indeed  as5rmptotically  reached.  We  could  not  prove  that  this  is  also  true 
when  the  vectors  are  circularly  arranged  but  not  mutxially  orthogonal.  How¬ 
ever,  it  could  not  disproved  by  any  numerical  simulation,  so  we  conjecture 
that  is  Indeed  true.  (Reasonable  selectivity  is  attained  even  in  most 
unfavorable  environments.  As  an  example,  in  a  circular  such  that  all 
cosines  fall  between  .94  and  1,  a  selectivity  of  .68  was  reached 

after  12,000  iterations.)  Notice  that  in  the  present  context  this 
question  is  only  of  theoretical  interest,  since  naturally  occuring  environ¬ 
ments  are  continuous  rather  than  discrete.  The  behavior  of  our  system  in 
such  an  environment  is  very  well  approximated  by  a  discrete  circular  d, 
provided  that  K  is  large  enough.  K  is  then  presumably  larger  than  N,  the 
K  inputs  are  linearly  dependent  and  we  have  no  explicit  formula  for  max  Sel^(m) . 

The  system  thus  functions  well  in  a  large  class  of  environments.  It  should 
be  stressed  that  the  numerical  value  of  the  only  parameter  that  appears 


explicitly  in  the  evolution  scheoe  Itself,  namely  the  exponent  p,  is 

not  at  all  critical.  Simulations  performed  with  a  constant  jJ,  p  being 

varied  from  .01  to  10,  yield  the  same  asymptotic  limit  for  the  selectivity; 

the  height  of  the  asymptotic  tuning  curve,  i.e.,  raax(m.d),  is  however 

a 

highly  dependent  on  p.  This  invariance  property  validates  in  a  sense 

the  definition  of  Sel.(x). 

d 

% 

Inhibitory  synapses  are  essential  here  exactly  as  they  are  in  the  2-dimensional 
case.  One  way  to  show  this  is  to  substitute  0  for  all  negative  components  in 
the  state  once  it  has  reached  selectivity.  This  typically  results  in  a 
drastic  drop  of  selectivity  (e.g.  from  .81  to  .55)  although  a  slight 
preference  generally  remains  for  the  same  orientation  as  before.  This  may 
be  related  to  the  experimental  fact  that  local  pharmacological  deactivation 
of  Inhibitory  connections  strongly  impairs  orientation  selectivity  by 
rendering  all  orientations  effective  in  triggering  the  cell’s  response 
(Silllto,  1975). 

Finally,  it  should  be  mentioned  that  the  system  displays  a  good  noise- 
tolerance,  particularly  when  the  state  has  already  reached  a  selective  region. 
The  system  then  resists  presynaptic  additive  noise  with  a  signal-to-noise 
ratio  of  the  order  of  1,  and  postsynaptic  noise  with  a  signal-to-noise  ratio 
as  small  as  1/4. 

Vb.  RESTRICTED  MONOCULAR  INPUT 

To  discuss  this  situation  we  must  now  Include  the  exponential  decay  term, 
-e;^i(t),  previously  neglected  (Eq.  4).  It  is  clear  that  the  results  stated 
above  will  be  preserved  if  e  is  sufficiently  smaller  than  the  average  of 


25. 


|^(c,c)|  (l.e.,  competition  mechanisms  are  faster  than  decay).  However, 
exponential  decay  may  become  crucial  if  one  asks  what  Is  the  response  of 
the  cell  to  patterns  that  were  not  represented  in  the  environment  during 
development . 


Consider  for  instance  an  environment  consisting  of  a  single  stimulus  d 
It  is  then  easily  shovm  that  system  6-7  admits  one  attractor  m^  that 


satisfies  m^*  =  c  for  small  e  (n^*  »  c  for  e 

o  o 


0) .  Obviously, 


for 


£  >  0,  will  satisfy  m^*  d  =  0  for  any  d  orthogonal  to  d^.  How¬ 


ever  the  response  to  a  pattern  d  not  orthogonal  to  d  will  depend  both  on 
e  and  on  cos(d,d^).  One  may  for  instance  find  that  d  =  1/2  (m^*  d^)  for 


cos(d,d  )  =  .5.  If  one  computes  now  the  selectivity  of  the  neuron  in  state 

1  1  K 

m  with  respect  to  a  circular  environment  (d  ,  ...,  d  )  such  that  min 

cos(d^,  d^)  =  .5,  one  will  obviously  get  a  low  value,  in  fact  Sel^(m^)<  .5. 


The  one-stimulus  environment  may  be  regarded  as  a  case  corresponding  to 
rearing  the  animal  in  a  visual  world  where  only  one  orientation  is  present. 

No  controversy  remains  at  present  that  rearing  in  such  a  visual  environment 
results  in  a  cortex  in  which  all  visually  responsive  cells  are  tuned  to  the 
experienced  (or  nearby)  orientations  (Blakemore  and  Cooper,  1970;  Hirsch  and 
Spinelli,  1970;  1971;  see  also  Stryker  et  al.,  1978).  We  see  that  our  theory 
is  in  agreement  with  these  findings;  moreover  we  predict  that  in  such  a  cortex 
the  average  selectivity  of  these  cells  should  be  lower  than  normal.  Although 
there  is  so  far  no  detailed  quantitative  study  on  this  point  in  a  recent  work 
there  is  some  indication  that  this  may  indeed  be  true:  "more  neurons  with 
normal  orientation  tuning  were  found  in  the  kittens  that  could  see  all 
orientations, or  at  least  horizontal  and  vertical,  than  in  the  kittens  that 
had  experienced  only  one  orientation"  (Rauschecker  and  Singer,  1980). 


Vc.  BINOCULAR  INPUT 


We  now  consider  a  blnocularly  driven  cell.  The  firing  rate  of  the 
neuron  at  time  t  becomes 

c(t)  =•  m^(t)*^^(t)  +  (8) 

with  evolution  schemes  for  "right"  and  "left"  states  m  and  m  straightforward 
generalizations  of  eq.  (4).  Various  p.-.ssibilities  now  exist  for  the  input 
(d^,  :  one  may  wish  to  simulate  normal  rearing  (both  and  circular 

and  presumably  highly  correlated),  monocular  deprivation,  binocular  de¬ 
privation  etc... 

Detailed  discussion  of  the  results  of  simulations  under  various  conditions 
is  given  in  the  next  section.  The  main  results  are  summarized  here: 

1.  In  a  normal  binocular  environment,  the  cell  becomes 
orientation  selective,  binocular,  preferring  the 
same  orientation  through  both  eyes. 

2.  In  an  environment  simulating  monocular  deprivation, 
the  cell  becomes  monocular  and  orientation-selective, 
whatever  its  initial  state. 

3.  In  environments  simulating  binocular  deprivation,  the 
cell  does  not  lose  its  responsiveness,  but  does  lose 
its  orientation  selectivity  if  it  was  present. 
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VI.  DEVELOPMENT  UNDER  DIFFERENT  REARING  CONDITIONS 
a)  Related  experimental  data 

This  brief  stnmnary  is  restricted  to  area  17  of  kitten’s  cortex.  Most 
kittens  first  open  their  eyes  at  the  end  of  the  first  week  after  birth. 

It  is  not  easy  to  assess  whether  or  not  orientation  selective  cells  exist 
at  that  time  in  striate  cortex:  few  cells  are  visually  responsive,  and 
the  response’s  main  characteristics  are  generally  "sluggishness"  and 
fatigability.  However,  it  is  quite  generally  agreed  that  as  soon  as 
cortical  cells  are  reliably  visually  stimulated  (e.g.  at  2  weeks),  some 
are  orientation  selective,  whatever  the  previous  visual  experience  of  the 
animal  (cf.  Hubei  and  Wiesel,  1963;  Blakemore  and  Van  Sluyters,  1975; 
Buisseret  and  Imbert,  1976;  Fregnac  and  Imbert,  1978). 

Orientation  selectivity  develops  and  extends  to  all  visual  cells  in  area 
17  if  the  animal  is  reared,  and  behaves  freely,  in  a  normal  visual 
environment  (NR) :  complete  "specification"  and  normal  binocularity  (about 
80%  of  responsive  cells)  are  reached  at  about  6  weeks  of  age  (Fi^gnac 
and  Imbert,  1978).  However,  if  the  animal  is  reared  in  total  darkness  from 
birth  to  the  age  of  6  weeks  (DR) ,  none  or  few  orientation  selective  cells 
are  then  recorded  (from  0  to  15%,  depending  on  the  authors  and  the 
classification  criteria) ;  however  the  distribution  of  ocular  dominance 
seems  unaffected  (Blakemore  and  Mitchell,  1973;  Imbert  and  Buisseret,  1975; 
Blakemore  and  Van  Sluyters,  1975;  Buisseret  and  Imbert,  1976;  Leventhal  and 
Hirsch,  1980;  Fregnac  and  Imbert,  1978).  In  animals  whose  eyelids  have  been 
sutured  at  birth,  and  which  are  thus  binocular ly  deprived  of  pattern  vision 
(BD) ,  a  somewhat  higher  proportion  (from  12  to  50%  of  the  visually  excitable 
cells  are  still  orientation  selective  at  6  weeks  (and  even  beyond  24  months  of 
age)  and  the  proportion  of  binocular  cells  is  less  than  normal  (Wiesel  and 


Hubei,  1965;  Blakemore  and  Van  Sluyters,  1975;  Kratz  and  Spear,  1976; 
Leventhal  and  Hlrsch,  1977,  Watkins  et  al.,  1978). 

Of  all  visual  deprivation  paradigms,  putting  one  eye  In  a  competitive 
advantage  over  the  other  has  probably  the  most  striking  consequences: 
monocular  lid-suture  (MD) ,  If  it  is  performed  during  a  "critical"  period 
(ranging  from  about  3  weeks  to  about  12  weeks) ,  results  in  a  rapid  loss 
of  binocularlty,  to  the  profit  of  the  open  eye  (Wiesel  and  Hubei,  1963, 
1965);  then  opening  the  closed  eye  and  closing  the  experienced  one  may 
result  in  a  complete  reversal  of  ocular  dominance  (Blakemore  and  Van 
Sluyters,  1974).  A  disruption  of  binocularlty  that  does  not  favor  one  of 
the  eyes  may  be  obtained,  for  example,  by  provoking  an  artificial 
strabismus  (Hubei  and  Wiesel,  1965)  or  by  an  alternating  monocular 
occlusion,  which  gives  both  eyes  an  equal  amount  of  visual  stimulation 
(Blakemore,  1976).  In  what  follows,  we  call  this  uncorrelated  rearing  (UR). 

b)  Simulations 

The  aim  of  this  section  is  to  show  that  the  experimental  results  briefly 
reviewed  above  follow  from  our  assumptions  if  one  chooses  the  appropriate 
distribution  for  The  model  system  now  consists  in  a  single  binocular 

neuron.  The  firing  rate  of  the  neuron  at  time  t  is  given  by 

^(t)  -  ;5i^(t)-5i^(t)+  ;^j^(t)-5jj^(t)  (8) 

where  the  Indices  r  and  Z  refer  to  right  and  left  eye  respectively. 

(resp.  )^j^)  obeys  the  evolution  scheme  described  by  equations  4-6,  where 
jJ^(resp.  is  substituted  for  The  two  equations  are  of  course  coupled, 
since  c(t)  depends  at  each  t  on  both  !(|^(t)  and  Jl“u^(t). 


The  vector  (5^,  Is  a  stationary  stochastic  process,  whose  distribution 
Is  one  of  the  following, depending  on  the  experimental  situation  one  wishes 


to  simulate: 


Normal  Rearing  (NR) i 

j^(t)  =  for  all  t,  and  Is  circular.  (Noise  terms  that  may  be 

added  to  the  inputs  may  or  may  not  be  stochastically  Independent) . 
Uncorrelated  Rearing  CUR) • 


^  and  are  i.i.d.  (independent  identically  distributed):  they  have 
the  same  circular  distribution,  but  no  statistical  relationship  exists 
between  them. 


Binocular  Deprivation: 

1.  Total  light  deprivation  (DR).  The  2n  components  of 

(^r,  i.i.d.:  ^  and  ^  are  uncorrelated  noise  terms. 

2.  Binocular  pattern  deprivation  (BD) .  )Jj.(t)  “  ^^(t)e, 

where  e  Is  an  arbitrary  fixed  vector  with 
positive  components,  and  and  are  scalar  posi  •  Ive-valued 
i.i.d. 


ttonocular  Deprivation  (MD) ; 

is  circular,  is  a  noise  term:  (Jy  =■  JJ 

In  the  NR  case,  the  Inputs  from  the  two  eyes  to  a  binocular  cell  are 
probably  well  correlated.  We  therefore  assume  that  they  are  equal,  which 
is  mathematically  equivalent.  The  DR  distribution  represents  dark  dis¬ 
charge.  The  BD  distribution  deserves  a  more  detailed  explanation.  In  this 
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distribution  it  is  only  the  length  and  of  the  vectors  d^  and  d^ 
that  varies  in  time.  This  length  is  thought  to  correspond  to  the 
Intensity  of  light  coming  through  each  closed  eyelid,  whereas  the 
direction  of  the  vector  in  the  input  space  is  determined  by  the  constant 
"unpat temed"  vector  e,  e.g.,  e  *  (1,1,  ...,  !)•  One  may  indeed  assume 
that  when  light  falls  on  the  retina  through  the  closed  lids,  there  is  at 
any  instant  of  time  high  correlation  between  firing  rates  of  all  retinal 
ganglion  cells  on  a  relatively  large  region  of  the  retina.  Inputs  from  the 
two  eyes,  however,  are  probably  to  some  extent  asynchronous  (cf.  Kratz 
and  Spear,  1976);  hence  the  BD  distribution. 

Simulations  of  the  behavior  of  the  system  in  these  different  environments 
give  the  following; 

NR  (Fig.  5a):  all  asymptotic  states  are  selective,  binocular 
with  matching  preferred  orientations  for  stimulation  through 
each  eye. 

uR  (Fig.  5b):  the  motion  of  the  state  (]^^,  m^  )  resembles  a 
random  walk.  (The  small  exponential  decay  term  is  necessary 
here  too,  in  order  to  prevent  large  fluctuations).  The 

g 

two  tuning  curves  therefore  undergo  random  fluctuations, 
that  are  essentially  determined  by  the  second-order  statistics 
of  the  input  d.  As  can  be  seen  from  the  figure,  these 
fluctuations  may  sometimes  result  in  a  weak  orientation 


preference  or  unbalanced  ocular  dominance.  However, 
the  system  never  stays  in  such  states  very  long;  its 
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average  state  on  the  long  run  is  perfectly  binocular  and 
non-oriented.  Moreover,  whatever  the  second-order  sta¬ 
tistics  of  and  the  circular  environment  in  which  tuning 
curves  are  assessed,  a  regular  unimodal  orientation  tuning 
curve  is  rarely  observed,  and  selectivity  never  exceeds  .6. 

We  may  thus  conclude  that  orientation  selectivity  as 
observed  in  the  NR  case  (both  experimental  and  theoretical) 
cannot  be  obtained  from  purely  random  synaptic  weights.  It 
is  worth  mentioning  here  that  prolonged  dark  rearing  has 
been  reported  to  increase  response  variability  (Leventhal 

and  Hirsch,  1980);  a  similar  observation  was  made  by 

^  9 

Fregnac  and  Bienenstock  (1981)  . 

BD  (Fig.  5c):  Unlike  the  DR  case,  the  state  converges 
(as  may  easily  be  proved  mathematically) .  Although  there 
exist  both  monocular  and  binocular  stable  equilibrium  points, 
the  asymptotic  state  is  generally  monocular  if  the  initial 
state  is  taken  as  0.  The  orientation  tuning  curve  is  then 
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essentially  determined  by  the  relative  geometry  of  the 
fixed  arbitrary  vector  e  and  the  arbitrary  circular  environ¬ 
ment  which  serves  to  assess  the  tuning  curve.  Fine  unimodal 
tuning  is  therefore  not  be  be  expected. 

MD(Fig.  5d) :  The  only  stable  equilibrium  points  are 
monocular  and  selective.  The  system  converges  to  such  states 
whatever  the  initial  conditions.  In  particular,  this  accounts 
for  reverse  suture  experiments  (Blakemore  and  Van  Sluyters, 

1974;  Movshon,  1976). 

UR(Fig.  5e) :  This  situation  is  in  a  sense  similar  to  the  BD 
one:  the  state  converges,  but  monocular  as  well  as  binocular 
equilibria  exist.  As  in  the  BD  case,  the  asymptotic  state 
generally  observed  with  m^(0)  =  m^^  (0)  *  0  is  monocular.  (This 
should  be  attributed  to  the  mismatched  inputs  from  the  two  eyes, 
as  is  done  by  most  authors).  In  this  case,  however,  asymptotic 
states  are  selective,  and  when  they  are  binocular,  preferred 
orientations  through  each  eye  don't  necessarily  coincide.  It 
should  be  mentioned  here  that  Blakemore  and  Van  Sluyters  (1974) 
report  that  after  a  period  of  alternating  monocular  occlusion, 
the  remaining  binocular  cells  may  differ  in  their  preferred 
orientations  for  stimulation  through  each  eye. 


f 
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These  results  are  in  agreement  with  classical  experimental  data  in 
the  domain  of  visual  cortex  development.  Further,  it  seems  likely  that 
we  can  also  explain  other  recent  data  obtained  from  more  complicated 
paradigms  such  as  those  reported  in  Rauschecker  and  Singer  C1981) • 

Most  of  the  results  presented  in  this  section  could  be  derived  fairly 
easily,  with  no  need  of  further  simulations,  from  our  knowledge  of  the 
convergence  to  selective  states  in  the  case  of  a  monocularly  driven  neuron 
in  a  circular  environment  (section  Va) .  Some  intriguing  properties  of 
our  theory,  however,  are  more  subtle.  As  an  example,  it  is  shown  in 
Appendix  C  that  in  the  MD  case,  the  degree  of  monocularity  of  the 
cortical  cell  is  correlated  with  its  orientation  selectivity  as  well  as 
the  diversity  of  inputs  to  the  open  eye.  These  unexpected  predictions 
agree  well  with  the  observation  by  Cynader  and  Mitchell  (1980)  and 
Trotter  et  al.  (in  press)  that  after  a  brief  period  of  monocular  exposure, 
oriented  cells  are  more  monocular  than  non-oriented  ones  as  well  as  the 
observation  of  Rauschecker  and  Singer  (1981)  that  an  open  eye  with  restricted 
inputs  leads  to  cells  oriented  to  the  restricted  input  that  arc  less 
monocularly  driven  than  usual. 

VII.  DISCUSSION 

We  propose  a  new  mathematical  form  for  synaptic  modification  and  in¬ 
vestigate  its  consequences  on  the  development  of  selectivity  in  cortical 
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neurons.  In  addition  we  provide  a  definition  of  the  notion  of  selectivity 
with  respect  to  a  random  variable  that  might  be  applied  In  many  different 
situations  (In  the  domain  of  development  of  sensory  systems,  for  example, 
selectivity  of  binocular  neurons  to  retinal  disparity,  etc.)  In  Its 
application  to  visual  cortex  our  theory  Is  In  agreement  with  classical 
experimental  results  obtained  over  the  last  generation  and  offers  a 
number  of  new  predictions,  some  of  which  can  be  tested  experimentally. 

This  may  lead  to  the  Identification  of  the  parameters  of  the  theory  and 
provide  Indications  as  to  the  biochemical  mechanisms  underlying  cortical 
plasticity. 

In  a  broader  context,  we  may  regard  our  form  of  synaptic  modification  as 
a  specific  correlation  modification  of  a  hebblan  type.  The  great  majority 
of  models  on  a  synaptic  level  In  domains  such  as  pattern  recognition, 
task  learning  or  associative  memory^*^  (which  are  less  accessible  to  direct 
neurophysiological  experimentation)  use  schemes  of  a  hebblan  type  with 
some  success.  We  are  thus  led  to  conjecture  that  some  form  of  correla¬ 
tion  modification  is  a  very  general  organizational  principle  that  manifests 
itself  in  visual  cortex  in  a  manner  that  is  accessible  to  experiment. 


Although  synaptic  competition  is  a  natural  consequence  of  Hebb’s  principle, 
Its  precise  form  must  be  defined.  A  distinction  was  made  In  Section  III 
between  spatial  competition — the  form  commonly  accepted  by  theoreticians 
as  well  as  experimentalists — and  temporal  competition,  the  form  utilized 
in  this  work.  The  performance  of  these  two  is  very  different  as  we  see 
most  clearly  in  the  development  of  selectivity.  In  the  temporal  version 
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asymptotic  states  are  of  maximum  selectivity  with  respect  to  the 
experienced  environment  (^,  Independent  of  the  geometry  of  This  was 
rigorously  proven  analytically  in  some  cases  (Theorem  1;  Section  IV) 
and  conjectured  on  the  basis  of  numerical  results  in  other  cases  (circular 
environments;  Section  V).  in  contrast  with  this,  we  claim  that  in  any 
model  using  spatial  competition,  correct  behavior  strongly  depends  on  the 
geometry  of  the  environment:  i.e.,  selective  states  are  not  reached  if  the 
patterns  in  the  environment  are  not  sufficiently  separated  from  one 
another.  Although  this  general  statement  is  not  proven  here,  a  proof 
is  sketched  in  Appendix  B  for  one  particular  model  using  spatial  competition 
between  converging  afferents.^^ 

We  further  note  that  selectivity,  as  was  shown  in  Section  VIb,  does  not 
develop  in  a  "pure  noise"  environment  (the  distribution  termed  DR).  Some 

kind  of  patterned  input  is  required.^^  It  follows  that,  even  at  the 
first  level  of  organization  of  connectivity,  information  is  being 
transferred  from  the  environment  to  the  system.  This  may  shed  some 
light  on  what  has  been  known  for  a  long  time  as  the  innate/leamed  con¬ 
troversy  in  visual  cortex.  Our  results  suggest  that  this  dichotomy  is, 
at  best,  misleading.  The  system’s  potential  developmental  ablllty~its 
evolutive  power  may  Indeed  be  genetically  determined;  yet  selectivity 
has  no  meaning  if  it  does  not  refer  to  a  given  structured  environment 
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that  determines  the  final  organization  of  the  system. 

The  present  work,  however,  makes  no  assumptions  concerning  the 
initial  state  of  cortex  (e.g.  the  presence  or  absence  of  selectivity 
at  eye-opening).  This  question,  still  a  subject  of  controversy  (see 
Pettigrew,  1978),  must  be  settled  experimentally.  Further,  although  we 
here  assume  that  all  synapses  are  equally  modifiable,  it  could  easily 
be  the  case  that  there  is  variation  in  modifiability — even  one  that  is 
time  dependent — and  that  for  example  some  of  the  initial  state  information 
is  contained  in  a  skeleton  of  synapses  that  is  less  modifiable. 

The  principal  results  of  our  theory,  applied  to  visual  cortical  neurons 
and  assuming  that  they  are  all  equally  modifiable  according  to 
equations  4-7  are  now  summarized.  These  results  are  either  in  agreement 
with  existing  experimental  data  or  are  new  and  somewhat  expected  con¬ 
sequences  of  our  theory. 


A.  Monocular ly  driven  neurons 

1)  A  monocularly  driven  neuron  in  a  'normal'  (pat¬ 
terned)  environment  becomes  selective.  The  precise  pattern  to  which  it 
becomes  selective  is  determined  at  random  if  the  initial  selectivity 

is  zero,  or  may  be  biased  toward  a  particular  pattern  if  there  is  a  built-in 
preference  for  this  pattern. 

2)  This  same  neuron  in  various  deprived  environments 


evolves  as  follows: 
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a)  Pure  noise.  The  neuron  becomes  less  selective 
but  continues  to  be  (somewhat)  responsive.  It  may  show  an  orientation 
preference,  but  this  is  relatively  unstable. 

b)  Exposure  to  a  single  pattern  (such  as  vertical 
lines) .  The  neuron  comes  respond  preferentially  to  the  single  pattern 
but  with  less  selectivity  (less  sharply  tuned)  than  if  all  orientations 
vere  present  in  the  environment .  This  last  is  a  natural  consequence  of 
temporal  competition  between  incoming  patterns  and  can  provide  a  good 
test  of  our  theory. 

3)  Inhibitory  synapses  are  required  to  produce 
maximum  selectivity.  If  such  inhibitory  connections  are  arbitrarily 
set  equal  to  zero,  selectivity  diminishes. 


B.  Binocular ly  Driven  Neurons 

1)  A  binocular ly  driven  neuron  in  a  'normal' 
(patterned)  environment  becomes  selective  and  binocular.  It  is  selctively 
driven  by  the  same  pattern  from  both  eyes. 

2)  This  same  binocularly  driven  neuron  in  various 
deprived  environments  evolves  as  follows: 

a)  Uncorrelated  patterned  inputs  to  both  eyes.  The 
neuron  becomes  selective,  often  monocularly  driven;  if  binocular  sometimes 
driven  by  different  patterns  from  the  two  eyes. 
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b)  Patterned  input  to  one  eye,  noise  to  the  other 
(monocular  deprivation) .  The  neuron  becomes  selective  and  generally  driven 
only  by  the  open  eye.  There  is  a  correlation  between  selectivity  and 
blnocularlty .  The  more  selective  the  neuron  becomes,  the  more  it  Is 
driven  only  by  the  open  eye.  A  non-selectlve  neuron  tends  to  remain 
binocularly  driven.  This  correlation  is  due  in  part  to  the  fact  that  it 
Is  the  same  mechanism  of  synaptic  change  that  serves  to  Increase  both  the 
selectivity  and  ocular  dominance  of  the  open  eye.  However  (as  shown 
in  Appendix  C)  there  is  also  a  subtler  connection;  It  is  the  non-preferred 


inputs  from  the  open  eye  accompanied  by  noise  from  the  closed  eye  that  drive 
the  neuron's  response  to  the  closed  eye  to  zero.  Thus,  for  example,  if  the 
visual  environment  were  such  that  there  were  mostly  preferred  inputs 


to  the  open  eye,  even  a  selective  cell  would  remain  less  monocular.  (It 
should  prefer  the  open  eye  but  remain  somewhat  driven  by  the  closed;) 

As  another  example,  a  kitten  dark-reared  to  the  age  of  about  ^2  days  (when 
there  remain  few  or  no  specific  cells)  and  then  given  monocular  exposure 
to  non-pattemed  input  would  retain  more  binocularly  driven  cells  than  a 
similar  animal  given  patterned  input. 

c)  Noise  input  to  both  eyes  (Dark  Rearing  or  Bi¬ 
nocular  Deprivation) .  The  neuron  remains  non-selective  (or  loses  its 


selectivity) and  diminishes  its  responsiveness  but  remains  binocularly  driven 
(in  contrast  to  the  situation  in  monocular  deprivation) . 


These  theoretical  conclusions  are  consistent  with  experimental  data  on 
Increases  and  decreases  in  selectivity,  data  concerning  changes  in  ocular 
dominance  in  various  rearing  conditions,  as  well  as  data  from  more  complicated 
paradigms.  Although  there  are  indications  in  recent  work  that  some  of  the 
new  predictions  are  in  agreement  with  experiment,  they  provide  the  opportionity 
for  tests  of  more  subtle  aspects  of  the  theory. 

In  conclusion,  we  note  that  a  precise  application  of  our  theory  to  certain 
complicated  experimental  situations  would  probably  require  inclusion  of 
some  anatomical  details,  intemeuronal  interactions,  as  well  as  a  statement 
of  what  information  is  innate  and  which  synapses  are  modifiable. 
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Appendix  Ar  Biochemical  Mechanism  for  Temporal  Competition 

It  is  probably  premature  to  propose  a  detailed  physiological  mechanism 
for  a  mathematical  synaptic  modification  algorithm:  too  many  possi¬ 
bilities  exist  with  no  present  experimental  test  to  decide  between  them. 
However,  we  propose  the  following  as  a  possible  example. 

The  dependence  of  our  modification  threshold  upon  the  mean  postsynaptic 
activity — which  regulates  the  individual  neuron  modification  in  an  overall 
manner — might  be  the  result  of  a  physiological  mechanism  within  the  frame¬ 
work  proposed  by  Changeux  et  al.  (1973).  Their  basic  hypothesis  is  that 
receptor  protein  on  the  postsynaptic  membrane  exists  in  two  states,  one 
labile,  the  other  stable;  selective  stabilization  of  the  receptor  takes 
place  during  development,  in  an  activity-dependent  fashion.  The  quantity 
of  labile  receptor  available  for  stabilization  is  determined  by  the  neuron's 
average  activity,  i.e.,  labile  receptor  is  not  synthesized  anymore  when  the 
neuron's  activity  is  high  for  a  relatively  long  period  of  time  (c>>c^) . 

(Cf.  Changeux  and  Danchin  (1976):  "The  activity  of  the  postsynaptic  ceil 
is  expected  to  regulate  the  synthesis  of  receptor".) 

Our  hypothesis  that  during  the  period  when  competition  really  takes  place 
(i.e.,  when  c  is  of  the  order  of  c^  in  eq.  7)  the  sign  of  the  modification 
is  determined  by  the  instantaneous  activity,  c,  relative  to  its  mean,  c, 
requires  that  a  single  message,  the  instantaneous  activity,  be  fed  back 
from  the  site  of  integration  of  the  incoming  message  to  the  individual 
synaptic  sites,  on  a  rapid  time-scale,  i.e.,  much  faster  than  the  one 
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involved  in  the  overall  regulation  mechanism.  This  might  be  contrasted 
with  the  assumption  implicit  in  most  spatial  competition  models,  namely, 
that  a  chemical  substance  is  redistributed  between  all  subsynaptic  sites 
(cf.  the  principle  of  conservation  of  total  synaptic  strength 
(von  der  Malsburg,  1973)). 
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Appendix  B:  Von  der  Malsburg's  model  of  development  of  orientation 
selectivity 


A  model  of  development  of  orientation  selectivity  using  an  evolution 

scheme  of  the  spatial  type  may  be  found  in  von  der  Malsburg  (1973).  We 

present  here  a  brief  analysis  of  this  model  in  viev  of  the  definition 

given  in  Section  III.  We  first  show  that  the  type  of  competition  implied 

by  this  model  is  indeed  formally,  the  spatial  one.  Next  we  investigate 

the  behavior  of  the  system  in  the  simple  situation  of  Theorem  1  in  Section  IV, 

i.e.,  for  a  two-pattern  environment,  the  dimension  of  the  system  being  N  =  2: 

We  will  show  why  the  assumption  that  is  made  of  nonlinearity  of  the  integrative 

power  is  a  necessary  one.  Finally,  we  prove  that  the  class  of  two-pattern 

environments  d  in  which  the  system  behaves  nicely,  i.e.,  the  state  is 

asymptotically  selective  with  respect  to  d,  is  defined  by  a  condition  of  the 
12  12 

type  0<cos(d  ,  d  )<a,  where  d  and  d  are  two  patterns  in  d,  and  a  is  a 
constant  strictly  less  than  1,  which  actually  depends  on  the  nonlinearity 
of  the  integrative  power,  i.e.,  on  its  threshold  0. 


For  the  purpose  of  our  analysis,  we  consider  a  single  "cortical"  neuron, 
whose  integrative  and  evolutive  power  are — in  our  notation — the  following: 


with 


c(t)  =  (?^(t)-5j(t))*, 

u-0ifu>0 

u* 

'0  if  u  <  0 


(Bl) 


(B2) 


mj(t+l) 


Y(t+l)(m.(t)+h  c(t)d,Ct)) 

"V  "Vj  -X,  'VJ 


N 


CB3) 


1 
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With  h  a  small  positive  constant  and  Y(t+1)  such  that: 


N  N 

r  p/t+l)  =  r  m  (t)  =  s  (B4) 

j-1  ^  j=l  ^ 


The  integrative  power  is  thus  nonlinear,  with  threshold  0.  The  normalizing 
factor  Y(t+1)  in  the  evolution  equation  (12),  keeps  the  sum  of  synaptic 
weights  constant,  equal  to  s.  All  variables  are  positive. 


Our  analysis  will  be  carried  out  on  this  reduced  version  of  von  der  Malsburg's 
model:  we  simply  ignore  the  fixed  Intracortical  connections  assumed  there, 
for  these  are  clearly  not  sufficient  to  tune  the  system  to  a  selective  state 
if  indlvldtial  neurons  don  * ;  already  display  this  property  as  is  clearly 
stated  by  the  author  himself,  the  ability  to  develop  selectivity  is  an  intrinsic 
property  of  individual  neurons,  the  intracortical  connections  being  there  to 
organize  orientation  perference  in  a  coherent  way  in  cortex.  (This  also  the 


viewpoint  in  the  present  work:  see  Appendix  D) .  Notice  that  this  is  by  no 
means  a  contradiction  to  the  fact  that  in  the  final  state,  Intracortical 
connections,  particularly  the  inhibitory  ones,  significantly  contribute 
to  the  selectivity  of  each  neuron. 


i 

i 


A  straightforward  calculation  shows  that  eqs.  B3  and  B4  are  equivalent  to 
the  following. 

.  J5j(t+l)-,gj(t)  -  j^(t)(jJj(t)/;J(t)  -  j[|j(t)/s)  j  -  1 . N  (B! 

'JC(t)  -  s  h  (^(t)j^(t)/(s  +  h  /^(t)jiJ(t)), 
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where  )J(t)  *  E  simulations,  }J(t)  is  actually  a  constant.) 


Thus,  according  to  eq.  B5,  the  sign  of  the  change  of  at  time  t  does  not 
depend  on  the  postsynaptic  activity  <^(t)  but  on  the  j-th  fiber  activity, 
d.(t).  This  is  clearly  spatial  competition,  as  was  intuitively  clear  from 
the  conservation  law  (eq.  B4). 


We  investigate  now  the  behavior  of  system  B5  in  a  two-pattem 
1  2 

environment:  Ptd=d  ]  *  1  "  *5-  For  this  purpose  we  slightly  modify 

the  original  setup:  there  the  dimension  is  relatively  high  (N=19)  but  the 

firing  frequencies  in  the  afferent  fibres  are  discrete-valued,  i.e., 

or  1,  j=l,...,N.  Here,  we  take  N=2  with  d^^  ^  allowed  to  take  any  value 

between  0  and  1.  We  thus  still  get  a  broad  range  of  environments 
1  2 

(cos(d  ,d  )  may  assume  any  value  between  0  and  1),  but  the  analysis  is 

made  considerably  easier.  To  further  simplify,  we  characterize  d  by  a 

12  12 

single  parameter  0<6<1  by  writing  d  =(1,6),  d  =(6,1).  Thus  cos(d  ,d  )  * 
26/(l+6^).  We  also  set  s  »  1. 

Under  these  circumstances,  averaging  the  evolution  equation  B5  with 
respect  to  d  leads  to  the  following: 

E[m,(t+1)  -  m,(t)]  =  h(2m,(t)-l)  (0(l+6)-26),  j-1,2.  (B6) 

J  J 

To  obtain  eq.  B6,  it  has  been  assumed  that  both  inputs  yield  above-threshold 

1  2 

responses,  i.e.,  m*d  and  m*d  >0.  Higher  order  terms  in  h  have  been  Ignored. 
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We  see  that  the  behavior  of  the  system  is  determined  by  the  sign  of 

the  quantity  0(l+6)-25.  Notice  that,  since  s*l,  0  cannot  be  arbitrarily 

1  2 

high:  in  order  that  states  m  exist  such  that  m"d  and  m*d  >0,  one  has  to 
assume  that  0<(l+6)/2. 

It  follows  from  eq.  B6  that  for  6  such  that  0(l+6)-26<O,  there  is  one 
attractor  of  selectivity  0,  namely  (.5;  .5).  When  6  gets  smaller  and 
0(l+6)-26  becomes  positive,  the  solution  bifurcates  into  two  attractors 
of  maximum  selectivity.  We  thus  conclude  that: 

1.  If  the  neuron's  Integrative  power  is  linear,  i.e. ,  0*0,  the 

1  2 

asymptotic  state  is  nonselective.  (When  0=0  and  d  and  d  are  orthogonal, 
(i.e., 6*0)  the  first-order  term  In  h  vanishes,  yet  the  second-order  term 
also  leads  to  the  nonselective  fixed  point.) 

2.  Given  a  fixed  O<0<1,  the  environments  that  are  acceptable  to 

the  system  are  those  which  satisfy  6<O/(2-0),  which  is  equivalent  to  a 

1  2 

condition  of  the  type  cos(d  ,d  )<a,  with  a  strictly  less  than  1.  (Notice 

that  in  the  actual  simulations,  d  consists  of  9  stimuli  that  are  indeed 

-v 

well  separated  from  one  another,  since  min j^cos(d^,d^)  =  1/7.) 


Appendix  C;  Correlation  Between  Ocular  Dominance  and  Selectivity  in  the 
Monocular  Deprived  Environment.  '  ^ 

Consider  the  MD  environment  In  Section  VIb;  It  is  defined  by  (j^,  jj)  where 
Is  circular  and  n  is  a  "pure  noise"  vector.  We  will  prove  that  the 
state  (m^,  0)  is  stable  in  this  environment  provided  that  m^  is  a  stable 
selective  state  in  the  environment  . 


^  ®  small  perturbation  from  equilibrium.  The  motion  at 

* 

point  (m^  is  given  by: 


(Clr) 

(CIO 


where  we  assume  that  the  noise  has  zero  mean. 

We  analyze  separately — somewhat  informally— the  behavior  of  the  two 

equations.  The  stability  of  eq.  Clr  is  immediate  from  the  stability  of 

* 

the  selective  state  m^  in  the  circular  enviornment  To  analyze  Cli, 

we  divide  the  range  of  the  right  eye  input  d  into  three 

"vr 

classes: 


(i)  d  is  such  that  m  •  d  is  either  far  above  threshold,  0„, 

r  r  'vr  M 

*  *  — 

and  therefore  <p(m  •  d  ,  m  •  d  )>0,or  far  below  threshold*  0^, 
r  f\^r  r  r  M 

(but  still  positive)  and  therefore  (l>(m*»d  .  m**d  )<0- 

r  ^r’  r  r  ’ 

(ii)  is  such  that  m^*  d^  is  near  threshold,  G^,  and 

therefore  i(i(m**  d  ,m*«d  )=0 
r  r  v 
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It 

(ill)  is  such  that  and  again 

*  *  _ 
jJr  »  ®r 


For  the  first  class  of  inputs,  the  sign  of  (f  is  determined  by  d 

'vr 

alone,  hence  Cl£  is  the  equation  of  a  random  walk.  To  Investigate  the 
behavior  of  Cli,  in  the  two  other  cases,  we  neglect  the  term  ijc^and 
linearize  (p  around  the  relevant  one  of  its  two  zeros.  It  is  easy  to 
see  that  case  (ii)  yields 


whereas  in  case  (lii)  one  obtains 


ii  “ 

where  and  £2  positive  constants,  measuring  respectively  the  absolute 
value  of  the  slope  of  p  at  the  modification  threshold  and  at  0. 


Since  n  is  a  noise-like  term,  its  distribution  is  presumably  symmetric 
with  respect  to  x.  and  averaging  eqs.  (C2)  and  (C3)  yields  respectively 


•  1  2 
"  N  ^1"  ^i 

-  N  "2 


where  n  is  the  average  squared  magnitude  of  the  noise  input  to  the  synapses 
from  the  closed  eye  and  N  is  the  dimensionality  of  the  system. 


We  thus  see  that  input  vectors  from  the  first  class  move  x  randomly.  Inputs 

'vi 

from  the  second  class  drive  it  away  from  0,  whereas  inputs  from  the  third 


U’ 
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drive  it  toward  0.  The  random  contribution  occurs  only  before  the  synaptic 

strengths  from  the  open  eye  have  settled  to  one  of  their  fixed  points. 

Once  these  strengths  have  reached  a  fixed  point,  only  (ii)  and  (iii)  occur. 

In  the  case  where  the  range  of  is  a  set  of  K  linearly  independent 
* 

vectors  and  m^  is  of  maximum  selectivity,  (K-l)/K,  case  (i)  does  not  occur 

1  *  1 

at  all.  Case  (ii)  occurs  only  for  one  input,  say  d^,  with  exactly 

equal  to  threshold,  0„,  and  (iii)  occurs  for  the  other  K-1  vectors  which 

M 

*  * 

are  all  orthogonal  to  m^.  The  more  selective  m^  with  respect  to  d^,  the 
higher  the  proportion  of  inputs  belonging  to  class  (iii),  the  class  that 
yields  eq.  C5,  l.e.,  that  brings  x  back  to  0. 


The  stability  of  the  global  system  still  depends  on  the  ratio  of  the 
quantities  and  as  well  as  on  the  statistics  of  the  noise  term  n 

'K, 

(e.g.  its  mean  square  norm).  We  may  however  formulate  two  general  con¬ 
clusions.  First,  under  reasonable  assumptions  of  the  order  of 
and  the  mean  square  norm  of  n  of  the  same  order  as  that  of  0 

is  stable  on  the  average  for  a  selective  m*.  Second,  the  residual  fluctuation 
of  around  0,  essentially  due  to  inputs  d^  in  classes  (i)  and  (ii),  is 
smaller  for  highly  selective  m*’s  than  it  is  for  mildly  selective  ones. 

Thus,  one  should  expect  that  in  a  monocularly  deprived  environment  non- 
selectlve  neurons  tend  to  remain  binocularly  driven.  In  addition  since  it 
is  the  non-preferred  inputs  from  the  open  eye  accompanied  by  noise  from  the 
closed  eye  (case  three)  that  drive  the  response  to  the  closed  eye  to  zero, 
if  Inputs  to  the  open  eye  were  restricted  to  preferred  inputs  (case  two)  even 
a  selective  cell  would  remain  less  monocular. 
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Appendix  D:  Many  Neuron  Systems 

It  is  very  likely  that  interactions  between  cortical  neurons  play  an 
important  role  in  overall  cortical  function  as  well,  perhaps,  as  in 
selectivity  of  individual  cortical  cells  (Creutzfeldt  et  al.,  1974  and 
Sillito,  1975).  Although  in  this  paper  we  have  treated  the  development 
of  selectivity  of  cortical  neurons  with  inputs  only  from  geniculate 
neurons,  the  input  space  can  be  generalized  to  include  Inputs  from  other 
cortical  neurons  as  well.  Thus  the  ideas  and  methods  employed  here  are 
equally  applicable  to  the  many-neuron  system.  Most  important,  the  result 
that  stable  equilibria  in  a  stationary  environment  are  selective  with 
respect  to  their  environment  can  be  taken  over  to  the  many-neuron  system. 

Consider  such  a  system  in  a  stationary  external  environment.  The  state 
of  each  cortical  neuron  now  has  two  parts;  one  relative  to  the  geniculo- 
cortical  synapses,  the  other  to  the  cortico-cortical  ones.  The  environ¬ 
ment  of  the  neuron  is  no  longer  stationary,  for  the  states  of  all  other 
cortical  neurons  in  the  system  evolve.  Yet,  when  the  system  reaches 
global  equilibrium,  which  will  occur  under  reasonable  assumptions,  each 
individual  environment  becomes  stationary.  The  single  unit  study  then 
allows  us  to  state  that,  at  least  in  principle,  (we  don’t  know  a  priori 
that  each  environment  is  circular)  the  state  of  each  neuron  is  selective 
with  respect  to  its  own  individual  environment. 

In  practice,  formulation  of  the  many-neuron  problem  poses  two  questions. 
First,  the  Integrative  power  of  the  system  should  be  specified.  Since 
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the  system  Includes  cortico-cortical  loops,  it  is  not  obvious  what  the 
response  to  a  given  afferent  message  should  be.  The  two  major  alternatives 
are:  a.  Stationary  cortical  activity  is  rapidly  reached,  (i.e.,  before 
the  afferent  message  changes);  b.  Relevant  cortical  activity  is  transitory. 
The  second  question  concerns  the  evolution  of  cortico-cortical  synaptic 
strengths;  shoul'^  these  synapses  be  regarded  as  modifiable  at  all  and 
if  yes,  how?  V  a  der  Malsburg  (1973),  assumes (a)  above  and  proposes  fixed 
connectivity  patterns,  short-range  excitatory,  longer-range  inhibitory. 

We  have  performed  a  simulation  of  a  many-neuron  system  using  the  much 
simplier  (and  probably  more  natural)  assumption  (b)  above:  1.  Only 
monosynaptically  and  disynaptically  mediated  components  of  the  afferent 
message  are  taken  into  account  for  the  computation  of  each  cortical  neuron’s 
activity,  before  modification  is  performed  and  a  new  stimulus  is  presented. 
2.  Intracortical  connections  are  fixed,  and  spatially  organized  as  in 
von  der  Malsburg  (1973).  3.  The  state  of  each  neuron  evolves  according 

to  eqs, 6  and  7  of  the  present  work.  The  results  are  the  following: 

1.  The  system's  state  converges.  2.  At  equilibrium,  each  neuron  stands 
in  a  selective  state  with  respect  to  the  environment,  3.  Preferred 
orientation — when  the  environment  is  a  circular  one — is  a  piecewise 
continuous  function  of  cortical  distance.  Point  3  is  Illustrated  in  Fig.  6. 


FOOTNOTES 


The  precise  form  of  the  averaging  integral,  i.e.  of  the 
convolution  kernel,  is  not  essential.  Exponential  kernels 
K(t)  *  exp(-t/T)  are  often  used  in  this  context  (see  e.  g. 

Nass  and  Cooper,  1975;  Uttley,  1976). 

The  mathematical  concept  that  is  needed  in  order  to  represent 
the  environment,  d, during  the  development  period  is  that  of  a 
stationary  stochastic  process, d  (t),  that  is  (roughly),  a  time- 
dependent  random  variable  whose  distribution  is  invariant  in  time. 

For  example,  d,  could  represent  an  elongated  bar  in  the  receptive 

a. 

field  of  the  neuron,  rotating  in  some  random  manner  around  its 
center.  At  each  instant,  the  probability  of  finding  the  bar  in 
any  given  orientation  is  the  same  as  at  any  other:  the  distribution  of 
^(t)  is  time-invariant,  uniform  over  the  interval  (0,  180°). 

Nonspecific  conditions  for  synaptic  decrease  such  as  uniform 
exponential  decay  is  clearly  insufficient  too:  in  Nass  and  Cooper 
(1975)  for  instance,  no  selectivity  is  achieved  without  lateral 
intracortical  inhibition.  Other  models  (von  der  Malsburg,  1973; 

Perez  et  al.,  1975)  use  a  normalization  rule  in  conjunction  with 
a  hebblan  scheme  for  synaptic  increase,  which  actually  results  in 
decrease  as  well  as  increase.  This  normalization  rule  is  discussed 
in  Appendix  B. 

Replacing  the  time  average  by  an  average  over  the  distribution  of 
is  allowed  provided  that  1)  the  process  d(t)  is  stationary,  2)  the 


interval,!, of  time  integration  is  short  with  respect  to  the 
process  of  synaptic  evolution  (l.e.,  changes  very  little 

during  an  interval  of  length  T) ,  3)  T  is  long  compared  to  the  mixing 
rate  of  the  process  (i.e.  during  a  period  of  length  T,  the  rela¬ 
tive  time  spent  by  the  process  ^(t)  at  any  point  d  in  the  input 
space  is  nearly  proportional  to  the  weight  of  the  distribution 
of  d  at  d).  Now,  synaptic  evolution  is  probably  a  slow  process, 
requiring  minutes  or  hours  (if  not  days)  to  be  significant,  whereas 
elementary  sensory  patterns  (e.g.  oriented  stimuli  in  the  receptive 
field  of  a  given  cortical  neuron)  are  nonnally  all  experienced  in 
an  interval  of  the  order  of,  say,  1  min.  or  less.  We  are  thus  able 
to  choose  T  so  that  a  good  estimate  of  c(t)  be  available  to  the  neuron. 

7.  The  sign  of  <J)(c,c)  for  c<0  is  not  crucial  since  c  is  essentially  a 
positive  quantity:  cortical  cells  in  general  have  low  spontaneous 
activity  and  at  any  rate  are  rarely  inhibited  much  below  their 
spontaneous  activity  level.  For  the  sake  of  mathematical  complete¬ 
ness,  one  may  however  wish  to  define  (j)(c,c)  for  negative  c;  (j)(c,c)>0 
is  then  the  most  convenient  for  it  allows  to  state  theorems  1-3  below 
under  the  most  general  initial  conditions. 

8.  The  circular  environment  which  serves  to  assess  the  orientation  tuning 
curves  is  now  in  a  sense  arbitrary,  since  it  is  not  at  all  used  in  the 
development  period.  The  same  remark  applies  of  course  to  the  BD  case. 

9.  In  fig.  IB  of  Fregnac  and  Bienenstock  (1981),  which  shows  averaged 
orientation  tuning  curves  of  a  cell  recorded  in  an  86  day  old  DR  cat, 
the  selectivity  is  .58  at  the  beginning  of  the  recording  session,  and 


.28  at  the  end. 


10.  Notice,  for  Instance,  the  analogy  between  states  of  maximum  selectivity 
as  defined  here  and  the  optimal  associative  mappings  of  Kohonen  (1977) . 

11-  More  complicated  temporal  or  mixed  spatio-temporal  schemes  are 

possible  and  some  such  have  been  proposed.  For  example,  Sejnowski 
(1977a)  has  suggested  a  form  of  modification  in  which  the  change  of 
the  j-th  S3maptic  strength  involves  the  covariance  between  the  j-feh 
fiber  and  post-synaptic  activities  (Sejnowski,  1977a).  In  addition, 
interaction  between  neurons  (such  as  lateral  inhibition)  can  in¬ 
crease  selectivity  (see  for  example,  Nass  and  Cooper  (1975)  and 
Appendix  D.) 

12.  Pure  noise  and  circular  environments  may  be  regarded  as  two 
extreme  cases:  the  first  totally  lacks  structure  whereas  the 
second  is  highly  organized.  Intermediate  cases,  i.e.,  environments 
consisting  of  the  sum  of  a  noise  process  and  of  a  circular  process 
have  also  been  Investigated  (see,  for  example,  Bienenstock  (1980)). 

There  it  is  shown  that  the  asymptotic  selectivity  directly  depends 
on  a  parameter  that  measures  the  degree  of  structure  of  the  environ¬ 
ment. 

13.  We  note,  further,  that  the  mechanism  of  synaptic  modification  we  have 
proposed  leads  both  to  what  are  sometimes  called  'selective'  and 
'instructive'  effects  (depending  on  the  structure  of  the  environment 

and  the  genetic  initial  state).  Thus,  as  is  already  suggested  by  Rauschecker 
and  Singer  (1981),  this  dichotomy  is  obscured,  or  does  not  appear  at  all,  at 


the  synaptic  level. 


14.  This  skeleton  might  consist  primarily  of  the  contralateral 
pathway  and  favor  the  development  of  orientation  preference  for 
horizontally  and  vertically  oriented  stimuli.  See,  for  example, 
Fregnac  (1979) . 

15.  In  addition  the  principle  of  temporal  competition  suggests  an 
experimental  paradigm  that  could  be  used  to  Increase  the  selectivity 
of  a  cortical  neuron  while  recording  from  the  same  neuron.  The 
paradigm  consists  of  controlling  the  postsynaptic  activity  of  the 
neuron  while  presenting  sequentially  in  its  receptive  field  two 
stimuli, A  and  B.  Stimulus  A (respectively  B)  should  be  associated  with 
a  high  (respectively  low)  instantaneous  firing  rate  in  such  a  way  as 
to  keep  the  cell's  mean  firing  rate  at  its  original  value.  We  predict 
that  the  cell  will  enventually  prefer  Stimulus  A  (i.e.  exhibit 
selectivity  with  respect  to  the  discrete  environment  consisting 

of  A  and  B) .  An  experiment  based  on  this  paradigm  is  currently 
being  undertaken  by  one  of  the  authors  (EB)  in  collaboration  with 
Yves  Fregnac  at  the  College  de  France. 

16.  In  this  situation  one  might  have  to  distinguish  between  short  and 
long  monocular  exposures.  In  very  long  monocular  exposures  the  decay 
term  of  eq.  4  (-em(t))  could  eventually  produce  decay  of  junctions 
from  the  closed  eye  independent  of  the  effect  discussed  above. 

17.  This  last  might  be  treated  as  for  example  in  Cooper,  Lleberman 
and  Oja  (1979). 


Figure  1:  Computing  the  selectivity  with  respect  to  an  environment 

uniformly  distributed  between  co  .  and  u 

min  max  _ 


The  abscissa  displays  a  parameter  of  the  stimulus,  e.g.  orientation 

(u)  -(!),=  180°),  the  ordinate  the  neuron's  response  0  is  the 

max  min 

level  of  average  spontaneous  activity;  M  is  the  maximum  response.  The 
selectivity  of  the  neuron  is  then  given  by 


Sel.  1-^(0)  -  (I)  .  ) 

%  max  min 


u 

r  max 

J  U)  . 
min 
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light  area 
total  box  area 


This  is  a  simple  measure  of  the  breadth  of  the  peak:  curves  of  same 
selectivity  have  approximately  the  same  half-width  at  half-height.  (Think 
for  Instance  of  triangularly  shaped  tuning  curves.) 


Typical  values  for  orientation  selectivity  of  adult  cortical  cells  vary 
between  0.7  and  0.85.  ("specific"  cells).  Selectivity  of  broadly  tuned 
but  still  unimodal  cells,  e.  g.  those  termed  "Immature"  by  Bulsseret 
and  Imbert  (1976)  and  Fregnac  and  Imbert  (1978),  lies  between  .5  and  .7. 
Obviously,  0  is  the  selectivity  of  an  absolutely  flat  curve,  whereas  1 
is  the  selectivity  of  a  Dirac  d-functlon. 

Figure  2:  A  function  satisfying  condition  (7). 

The  3  diagrams  show  the  behavior  of  ())(c,c)  as  a  function  of  c  for  three  different 
constant  values  of  c.  In  each  diagram,  the  solid  part  of  the  curve  repre- 


sents  4>(c,c)  in  the  vicinity  of  c,  which  of  course  is  the  relevant 

part  of  this  function.  In  the  upper  diagram  (c>>Cq),  although  <t»(c,c)  is  not 

negative  for  all  c  as  was  formally  required  (see  text),  the  probability 

that  <|)(c,c)  >  0  is  small  and  gets  even  smaller  as  c  Increases.  The  important 

point  in  the  definition  of  is  the  nonlinearity  of  which  makes  it 

increase  or  decrease  faster  than  c,  while  6,,(c)  is  of  the  same  order  as 

M 

c,  if  c  itself  is  of  the  same  order  as  c^. 

Figure  3:  The  phasj  portrait  of  Eq.  6  in  an  environment  consisting  of  2 
_ inputs  d  and  d  (Theorems  1  and  2). _ 

The  diagram  shows  the  trajectories  of  the  state  of  the  system,  starting 

from  different  initial  points.  This  is  a  computer  simulation  performed 

with  one  given  function  (fi  satisfying  condition  7.  Using  a  different 

function  may  slightly  change  the  shape  of  the  trajectories,  without  any 

1  2 

essential  change  in  the  behavior.  The  unstable  fixed  points  are  m  ’ 

0  12 

and  m  ,  the  stable  ones  m  and  m  .  The  system  is  a  stochastic  one,  which 

means  that  the  trajectories  depend  in  fact  on  the  precise  sequence  of 

inputs.  As  long  as  the  state  Is  in  the  unshaded  region,  it  is  not  yet 

1  2 

known  whether  it  will  eventually  be  attracted  to  m  or  m  .  This  be 

comes  determined  as  the  state  enters  one  of  the  trap  (shaded)  regions,  or  F 

Trajectories  shown  here  are  deterministic  ones,  obtained  by  alternating  d 

12  ^ 
regularly  between  d  and  d  .  They  are  in  fact  the  averaged  trajectories 

of  the  state  and  are  much  more  regular  and  smooth  than  the  actual 

stochastic  ones. 

Fieure  4;  The  evolution  of  a  synaptic  system  in  a  circular  environment. 


Here,  K  *  40,  and  the  vectors  are  linearly  dependent.  The  value  of  the 
maximum  selectivity  with  respect  to  d  is  therefore  not  precisely 


calculable.  The  asumptotic  selectivity  is  approximately  .9,  perhaps 
the  maximum  selectivity,  (a)  demonstrates  the  progressive  buildup  of 
the  selectivity  in  a  circular  environment  d  while  (b)  shows  the  resulting 
tuning  curve  at  t  =  1000. 

Figure  5;  Results  of  computer  simulations  corresponding  to  various 
_ rearing  conditions. _ 

In  these  simulations  the  upper  and  lower  figures  show  cell  responses  to 
stimuli  from  the  two  eyes. 

a)  Normal  (NR).  The  cells  response  is  binocular  and  selective. 

b)  Dark  rearing  (DR).  No  stable  selectivity  in  the  cell’s  response. 

The  response  curve  flucturates  randomly.  The  cell  is  on  the 
average  binocularly  driven. 

c)  Binocular  deprivation  (BD) .  The  cell  reaches  a  final  state 
corresponding  to  the  arbitrary  vector  corresponding  to  a  diffuse 
input  to  the  retina.  The  cells  are  sometimes  monocular ly  driven. 
This  is  somewhat  analogous  to  (e)  below. 

d)  Monocular  deprivation  (MD) .  The  cell’s  response  is  monocular  and 
selective. 

e)  Uncorrelated  Rearing  (UR).  Both  binocular  and  monocular  selective 
final  states  are  observed. 


Figure  6:  A  regular  distribution  of  preferred  orientation  In  a  one- 
_ dimensional  cortex. _ 

The  system  Is  an' array  of  50  "cortical"  cells  arranged  in  a  cyclic  way 
(cell  no.  1  and  cell  no.  50  are  neighbors)  and  interconnected  according  to  a 
fixed  short-range-excitation-long-range-inhibition  pattern.  (Coefficients 
of  interactions  are,  as  a  function  of  increasing  inter-cell  distance: 

.4, .4, -.2, -.4, -.4,0,  ...).  The  environment  d  of  the  system  is  the  usual 
circular  one.  Integrative  and  evolutive  powers  are  described  in  the 
text.  When  the  system  reaches  equilibrium,  one  has  .73<Sel.(m. )<.77  for 

-  ^  i  - 

all  i  between  1  and  50.  The  diagram  shows  preferred  orientation  as  a 
function  of  cortical  coordinate. 
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